Results

Intraclass Correlation

Intraclass Correlation
Type Point Estimate Lower 95% CI Upper 95% CI
ICC3,1 0.209 0.150 0.285
Note.  100 subjects and 10 raters/measurements. ICC type as referenced by Shrout & Fleiss (1979).

Repeated Measures ANOVA

Within Subjects Effects
Cases Sphericity Correction Sum of Squares df Mean Square F p ω²
RM Factor 1 None 509,686 15.00 33,979.0 77.63 < .001 0.379
Residuals None 650,015 1,485.00 437.7      
Note.  Sphericity corrections not available for factors with 2 levels.
Note.  Type III Sum of Squares
Between Subjects Effects
Cases Sum of Squares df Mean Square F p
Residuals 166,042 99 1,677    
Note.  Type III Sum of Squares

Descriptives

Raincloud plots

Dependent

Assumption Checks

Test of Sphericity
Cannot perform sphericity tests because there are only two levels of the RM factor, or because the SSP matrix is singular.
  Mauchly's W Approx. Χ² df p-value Greenhouse-Geisser ε Huynh-Feldt ε Lower Bound ε
. . . . . . . .

Nonparametrics

Friedman Test
Factor Χ²F df p Kendall's W
RM Factor 1 671.9 15 < .001 0.448

Conover Test

Conover's Post Hoc Comparisons - RM Factor 1
    T-Stat df Wi Wj rrb p pbonf pholm
1 2 0.571 1,485 1,108.0 1,079.5 -0.100 .568 1.000 1.000
  3 13.376 1,485 1,108.0 440.5 0.857 < .001 < .001 < .001
  4 3.717 1,485 1,108.0 922.5 0.279 < .001 .025 .007
  5 12.675 1,485 1,108.0 475.5 0.956 < .001 < .001 < .001
  6 5.260 1,485 1,108.0 845.5 0.475 < .001 < .001 < .001
  7 5.691 1,485 1,108.0 824.0 0.119 < .001 < .001 < .001
  8 10.310 1,485 1,108.0 593.5 0.905 < .001 < .001 < .001
  9 1.753 1,485 1,108.0 1,195.5 -0.522 .080 1.000 1.000
  10 6.333 1,485 1,108.0 792.0 0.519 < .001 < .001 < .001
  Mean 3.787 1,485 1,108.0 919.0 0.709 < .001 .019 .005
  Median 5.872 1,485 1,108.0 815.0 0.833 < .001 < .001 < .001
  stud-prof 17.414 1,485 1,108.0 239.0 0.927 < .001 < .001 < .001
  stud-stud 7.825 1,485 1,108.0 717.5 0.498 < .001 < .001 < .001
  stud-obj 1.122 1,485 1,108.0 1,164.0 -0.430 .262 1.000 1.000
  estimated 7.234 1,485 1,108.0 1,469.0 -0.973 < .001 < .001 < .001
2 3 12.805 1,485 1,079.5 440.5 0.887 < .001 < .001 < .001
  4 3.146 1,485 1,079.5 922.5 0.476 .002 .202 .051
  5 12.104 1,485 1,079.5 475.5 0.961 < .001 < .001 < .001
  6 4.689 1,485 1,079.5 845.5 0.419 < .001 < .001 < .001
  7 5.120 1,485 1,079.5 824.0 0.165 < .001 < .001 < .001
  8 9.739 1,485 1,079.5 593.5 0.889 < .001 < .001 < .001
  9 2.325 1,485 1,079.5 1,195.5 -0.402 .020 1.000 .465
  10 5.761 1,485 1,079.5 792.0 0.559 < .001 < .001 < .001
  Mean 3.216 1,485 1,079.5 919.0 0.702 .001 .159 .041
  Median 5.300 1,485 1,079.5 815.0 0.856 < .001 < .001 < .001
  stud-prof 16.843 1,485 1,079.5 239.0 0.913 < .001 < .001 < .001
  stud-stud 7.254 1,485 1,079.5 717.5 0.514 < .001 < .001 < .001
  stud-obj 1.693 1,485 1,079.5 1,164.0 -0.360 .091 1.000 1.000
  estimated 7.805 1,485 1,079.5 1,469.0 -0.956 < .001 < .001 < .001
3 4 9.659 1,485 440.5 922.5 -0.833 < .001 < .001 < .001
  5 0.701 1,485 440.5 475.5 -0.250 .483 1.000 1.000
  6 8.116 1,485 440.5 845.5 -0.779 < .001 < .001 < .001
  7 7.685 1,485 440.5 824.0 -0.579 < .001 < .001 < .001
  8 3.066 1,485 440.5 593.5 -0.529 .002 .265 .064
  9 15.130 1,485 440.5 1,195.5 -0.957 < .001 < .001 < .001
  10 7.044 1,485 440.5 792.0 -0.872 < .001 < .001 < .001
  Mean 9.589 1,485 440.5 919.0 -0.842 < .001 < .001 < .001
  Median 7.505 1,485 440.5 815.0 -0.820 < .001 < .001 < .001
  stud-prof 4.038 1,485 440.5 239.0 0.730 < .001 .007 .002
  stud-stud 5.551 1,485 440.5 717.5 -0.445 < .001 < .001 < .001
  stud-obj 14.499 1,485 440.5 1,164.0 -0.951 < .001 < .001 < .001
  estimated 20.611 1,485 440.5 1,469.0 -0.992 < .001 < .001 < .001
4 5 8.958 1,485 922.5 475.5 0.850 < .001 < .001 < .001
  6 1.543 1,485 922.5 845.5 0.148 .123 1.000 1.000
  7 1.974 1,485 922.5 824.0 -0.006 .049 1.000 .923
  8 6.593 1,485 922.5 593.5 0.814 < .001 < .001 < .001
  9 5.471 1,485 922.5 1,195.5 -0.694 < .001 < .001 < .001
  10 2.615 1,485 922.5 792.0 0.257 .009 1.000 .252
  Mean 0.070 1,485 922.5 919.0 0.198 .944 1.000 1.000
  Median 2.154 1,485 922.5 815.0 0.409 .031 1.000 .690
  stud-prof 13.697 1,485 922.5 239.0 0.868 < .001 < .001 < .001
  stud-stud 4.108 1,485 922.5 717.5 0.244 < .001 .005 .002
  stud-obj 4.840 1,485 922.5 1,164.0 -0.677 < .001 < .001 < .001
  estimated 10.952 1,485 922.5 1,469.0 -0.990 < .001 < .001 < .001
5 6 7.415 1,485 475.5 845.5 -0.657 < .001 < .001 < .001
  7 6.984 1,485 475.5 824.0 -0.768 < .001 < .001 < .001
  8 2.365 1,485 475.5 593.5 -0.475 .018 1.000 .436
  9 14.428 1,485 475.5 1,195.5 -0.925 < .001 < .001 < .001
  10 6.343 1,485 475.5 792.0 -0.589 < .001 < .001 < .001
  Mean 8.888 1,485 475.5 919.0 -0.912 < .001 < .001 < .001
  Median 6.803 1,485 475.5 815.0 -0.880 < .001 < .001 < .001
  stud-prof 4.739 1,485 475.5 239.0 0.692 < .001 < .001 < .001
  stud-stud 4.850 1,485 475.5 717.5 -0.584 < .001 < .001 < .001
  stud-obj 13.797 1,485 475.5 1,164.0 -0.881 < .001 < .001 < .001
  estimated 19.909 1,485 475.5 1,469.0 -0.999 < .001 < .001 < .001
6 7 0.431 1,485 845.5 824.0 -0.243 .667 1.000 1.000
  8 5.050 1,485 845.5 593.5 0.410 < .001 < .001 < .001
  9 7.014 1,485 845.5 1,195.5 -0.638 < .001 < .001 < .001
  10 1.072 1,485 845.5 792.0 0.046 .284 1.000 1.000
  Mean 1.473 1,485 845.5 919.0 -0.045 .141 1.000 1.000
  Median 0.611 1,485 845.5 815.0 0.179 .541 1.000 1.000
  stud-prof 12.154 1,485 845.5 239.0 0.893 < .001 < .001 < .001
  stud-stud 2.565 1,485 845.5 717.5 0.155 .010 1.000 .281
  stud-obj 6.383 1,485 845.5 1,164.0 -0.558 < .001 < .001 < .001
  estimated 12.495 1,485 845.5 1,469.0 -0.952 < .001 < .001 < .001
7 8 4.619 1,485 824.0 593.5 0.442 < .001 < .001 < .001
  9 7.445 1,485 824.0 1,195.5 -0.412 < .001 < .001 < .001
  10 0.641 1,485 824.0 792.0 0.161 .521 1.000 1.000
  Mean 1.904 1,485 824.0 919.0 0.107 .057 1.000 .971
  Median 0.180 1,485 824.0 815.0 0.212 .857 1.000 1.000
  stud-prof 11.723 1,485 824.0 239.0 0.885 < .001 < .001 < .001
  stud-stud 2.134 1,485 824.0 717.5 0.439 .033 1.000 .693
  stud-obj 6.813 1,485 824.0 1,164.0 -0.280 < .001 < .001 < .001
  estimated 12.926 1,485 824.0 1,469.0 -0.911 < .001 < .001 < .001
8 9 12.064 1,485 593.5 1,195.5 -0.885 < .001 < .001 < .001
  10 3.978 1,485 593.5 792.0 -0.334 < .001 .009 .002
  Mean 6.523 1,485 593.5 919.0 -0.779 < .001 < .001 < .001
  Median 4.439 1,485 593.5 815.0 -0.701 < .001 .001 < .001
  stud-prof 7.104 1,485 593.5 239.0 0.780 < .001 < .001 < .001
  stud-stud 2.485 1,485 593.5 717.5 -0.225 .013 1.000 .327
  stud-obj 11.433 1,485 593.5 1,164.0 -0.867 < .001 < .001 < .001
  estimated 17.545 1,485 593.5 1,469.0 -0.999 < .001 < .001 < .001
9 10 8.086 1,485 1,195.5 792.0 0.800 < .001 < .001 < .001
  Mean 5.541 1,485 1,195.5 919.0 0.828 < .001 < .001 < .001
  Median 7.625 1,485 1,195.5 815.0 0.875 < .001 < .001 < .001
  stud-prof 19.168 1,485 1,195.5 239.0 0.961 < .001 < .001 < .001
  stud-stud 9.579 1,485 1,195.5 717.5 0.628 < .001 < .001 < .001
  stud-obj 0.631 1,485 1,195.5 1,164.0 -0.049 .528 1.000 1.000
  estimated 5.481 1,485 1,195.5 1,469.0 -0.631 < .001 < .001 < .001
10 Mean 2.545 1,485 792.0 919.0 -0.187 .011 1.000 .287
  Median 0.461 1,485 792.0 815.0 -0.036 .645 1.000 1.000
  stud-prof 11.082 1,485 792.0 239.0 0.873 < .001 < .001 < .001
  stud-stud 1.493 1,485 792.0 717.5 0.089 .136 1.000 1.000
  stud-obj 7.455 1,485 792.0 1,164.0 -0.715 < .001 < .001 < .001
  estimated 13.567 1,485 792.0 1,469.0 -0.957 < .001 < .001 < .001
Mean Median 2.084 1,485 919.0 815.0 0.656 .037 1.000 .746
  stud-prof 13.627 1,485 919.0 239.0 0.879 < .001 < .001 < .001
  stud-stud 4.038 1,485 919.0 717.5 0.197 < .001 .007 .002
  stud-obj 4.910 1,485 919.0 1,164.0 -0.696 < .001 < .001 < .001
  estimated 11.022 1,485 919.0 1,469.0 -0.994 < .001 < .001 < .001
Median stud-prof 11.543 1,485 815.0 239.0 0.863 < .001 < .001 < .001
  stud-stud 1.954 1,485 815.0 717.5 0.087 .051 1.000 .923
  stud-obj 6.994 1,485 815.0 1,164.0 -0.724 < .001 < .001 < .001
  estimated 13.106 1,485 815.0 1,469.0 -0.994 < .001 < .001 < .001
stud-prof stud-stud 9.589 1,485 239.0 717.5 -0.818 < .001 < .001 < .001
  stud-obj 18.537 1,485 239.0 1,164.0 -0.895 < .001 < .001 < .001
  estimated 24.649 1,485 239.0 1,469.0 -1.000 < .001 < .001 < .001
stud-stud stud-obj 8.948 1,485 717.5 1,164.0 -0.524 < .001 < .001 < .001
  estimated 15.060 1,485 717.5 1,469.0 -1.000 < .001 < .001 < .001
stud-obj estimated 6.112 1,485 1,164.0 1,469.0 -0.469 < .001 < .001 < .001
Note.  Grouped by subject.
Note.  Rank-biserial correlation based on individual signed-rank tests.

Paired Samples T-Test

Paired Samples T-Test
Measure 1   Measure 2 Test Statistic z df p Effect Size SE Effect Size
Median - stud-stud Student 0.201   99 .841 0.020 0.118
Wilcoxon 2,690.500 0.752 .453 0.087 0.115
Median - stud-obj Student -7.959   99 < .001 -0.796 0.177
Wilcoxon 698.000 -6.282 < .001 -0.724 0.115
Median - stud-prof Student 10.518   99 < .001 1.052 0.185
Wilcoxon 4,610.000 7.452 < .001 0.863 0.115
Median - interaction-estimated Student -22.369   99 < .001 -2.237 0.230
Wilcoxon 16.000 -8.627 < .001 -0.994 0.115
Mean - stud-stud Student 1.527   99 .130 0.153 0.113
Wilcoxon 3,022.000 1.709 .088 0.197 0.115
Mean - stud-obj Student -7.083   99 < .001 -0.708 0.176
Wilcoxon 768.000 -6.041 < .001 -0.696 0.115
Mean - stud-prof Student 13.195   99 < .001 1.319 0.203
Wilcoxon 4,744.000 7.630 < .001 0.879 0.115
Mean - interaction-estimated Student -22.610   99 < .001 -2.261 0.223
Wilcoxon 14.000 -8.634 < .001 -0.994 0.115
Note.  For the Student t-test, effect size is given by Cohen's d . For the Wilcoxon test, effect size is given by the matched rank biserial correlation.

Descriptives

Raincloud Plots

Median - stud-stud
Median - stud-obj
Median - stud-prof
Median - interaction-estimated
Mean - stud-stud
Mean - stud-obj
Mean - stud-prof
Mean - interaction-estimated

Raincloud Difference Plots

Median - stud-stud
Median - stud-obj
Median - stud-prof
Median - interaction-estimated
Mean - stud-stud
Mean - stud-obj
Mean - stud-prof
Mean - interaction-estimated